The invention generally relates to the automatic recognition of gestures, performed by persons, by use of a computer (gesture recognition). In particular, the invention relates to a gesture interaction with a driver information system of a vehicle.
Within the context of a human-computer interaction, Kurtenbach et al. (“Gestures in Human-Computer Communication” in “The Art of Human-Computer Interface Design”, pp. 309-317, 1990) define a gesture as a movement of a body which contains information. For example, waving at a departure is a gesture. In contrast, pressing a key on a keyboard is not a gesture since the movement of the finger on the way to the collision with the key is neither observed nor is significant since it is only of relevance which key has been pressed.
In vehicles, it is already known to perform an operation of vehicle components by means of an interpretation of gestures. DE 42 01 934 A1 shows a data processing system which recognizes gestures and translates these into commands for controlling the data processing system. EP 1 408 443 A1 also shows a system in which control of a function takes place via a gesture recognition which, for example, includes various gestures in the form of hand or finger positions which are detected by way of a 2D or 3D camera. The respective hand and finger positions are allocated corresponding actions. The interpretation utilizes both static gestures and also particular sequences of movements.
A further approach is the recognition of gestures by way of stereoscopy. EP 1 477 924 A2 shows, for example, a gesture recognition device for recognizing attitudes or gestures of a person on the basis of images of the person which have been recorded by two cameras.
Modern vehicles provide the vehicle driver with extensive information and control options, for example via a central display screen. In particular, the operation of modern navigation systems is a continuous subject matter of research in order to improve the operation by the vehicle driver particularly during the trip so that the vehicle driver is distracted as little as possible from the actual driving of the vehicle and his attention for the environment and the road traffic is impaired as little as possible. However, the control options by use of gesture recognition, described before, are frequently not very intuitive. For this reason, the attempt at simplification of operation of a vehicle component can frequently not be achieved since gestures of the driver are not always translated into the operating inputs intended by the driver or are not recognized at all. The consequence of this is that the actual operating process is longer than theoretically expected and the attention of the driver is distracted more.
It is an object of the present invention to provide an improved control option, particularly for controlling a map section of a navigation system or screen contents of a driver information system, for example of an implemented Internet browser, which overcomes the disadvantages in the prior art.
This and other objects are achieved in accordance with embodiments of the invention.
A core concept of the invention consists in controlling, for example moving and/or magnifying, a displayed map section of a navigation map of a navigation system or a comparable movable content on a central display unit of a driver information system via a gesture, preferably free-space gestures. By means of a special gesture, e.g. a pincer grip with two fingers, the display content on the central display can be adjusted in three dimensions (3D) in accordance with the position of a camera virtually available and movable in free space.
By way of the control system according to the invention, it is achieved that an intuitive operating sensation is conveyed to the driver or an operator due to a special relationship between the hand movement and the change of the display content. This simplifies the control of the navigation system or of the driver information system and makes it more reliable so that especially the driver is impaired less by an operating process in the driving of the vehicle.
The invention thus relates to a control system for moving and/or magnifying a display content of a display unit. The control system includes: (i) a display unit having display content, at least one camera which is designed to record a sensing region in front of the display unit, (ii) a gesture recognition unit which is coupled to the at least one camera and is designed to recognize a predetermined gesture performed with a hand and a current position of the gesture in the sensing region, and (iii) a display content adapting unit which is designed to adapt the display content in accordance with a change of the current position of the gesture, in particular to move the display content accordingly in the event of a change of the position in a plane parallel to the display unit and/or to enlarge or to reduce the size of the display content in the event of a change of the position toward or away from the display unit.
The control system can have two cameras for recording the sensing region in front of the display unit. The gesture recognition unit can thus determine the current position of the gesture in the sensing region by way of stereoscopy.
The at least one camera can be an infrared camera, a time-of-flight camera and/or a structured-light camera.
The actual gesture recognition can take place by way of a continuous sequence of images of a hand/finger position of a user by use of technical image analysis. In this context, two approaches are basically possible: the gesture recognition unit can contain a database with examples of relevant gestures which have been generated by way of a meridian of a multiplicity, for example via 1000 video analyses of various embodiments of the gesture. An optically detected gesture can then be compared with the database by way of image recognition algorithms and the relevant gesture can be recognized on the basis of the comparison in order to trigger the associated operating function. As an alternative, it is also possible to work with a so-called skeleton recognition wherein the hand and/or fingers are recognized in the image data and by use of a simplified skeleton model, predefined gestures are inferred.
In the actual recognition of the gestures, the image information is processed by image processing algorithms, known per se, which analyze the raw data and finally recognize the gestures. In this context, algorithms for pattern recognition can be used. To remove noise in the input data and for reducing the data, preprocessing of the image data can take place in a first step. Subsequently, features are extracted from the image data in order to recognize the gesture by way of a classification. For this purpose, it is possible to use, for example, Hidden Markov models, artificial neural networks and other techniques, known per se from research in artificial intelligence.
The gesture recognition unit and/or the display content adapting unit is or are preferably designed to move and/or to magnify the display content as long as the predetermined gesture is recognized. The sensing region is preferably a predetermined space in front of the display unit.
The display content adapting unit can be designed to enlarge or to reduce the size of the display content in the case of a change of the position of the gesture toward or away from the display unit by a factor. In this context, the factor preferably corresponds to a power of base 2 with the change of the distance between the position of the gesture and the display unit as exponent. By means of the power of 2 with the change of distance as exponent it is achieved that a doubling or halving of the display content shown is achieved via a defined movement of the gesture or hand of the user in the x direction. Thus, it is possible to change between very many magnifying factors with a single hand movement. The exponent can also be scaled additionally with a sensitivity factor.
The display content adapting unit can be designed to move the display content with a change in the position of the gesture in a plane in parallel to the display unit correspondingly in such a manner that a length of a movement of the display content corresponds to a distance of the movement of the gesture in the plane parallel to the display unit. That is, a hand movement over a distance of 10 cm in a plane parallel to the display unit to the right, also results in a movement of the display content by 10 cm to the right. This results in a perfect mental operating model for a user as a result of which the control is intuitive for a user.
The display content adapting unit can also be designed, as soon as the coordinates of the last position of the gesture have been acquired, to allow the display content to continue to run in accordance with a last sensed speed or the last speed sensed at which the position of the gesture has changed, and to slow down preferably continuously until it comes to a standstill. This control characteristic conveys to a user the feeling of being able to virtually bump or nudge the display content by way of the predetermined gesture.
The display content is preferably a map section of a map, e.g. a navigation map of a navigation system. The map data and thus the map can be stored, for example, in a database of the navigation system. The display content adapting unit is then used for adapting the map section in accordance with the change in the current position of the gesture.
The invention also relates to a vehicle, particularly a motor vehicle, having a driver information system which has at least one display unit for displaying graphical data, for example of a navigation system or of an Internet browser, and a control system according to the invention as described above, for moving and/or magnifying the display content.
The invention is particularly well suited for controlling display contents of a driver information system in a vehicle. However, the invention can also be basically implemented on smartphones, tablet computers or devices having a display, the content of which can be adapted by a user in accordance with the control system proposed here.
Further advantages, features and details of the invention can be obtained from the subsequent description in which exemplary embodiments of the invention are described in detail with reference to the drawings. Similarly, the aforementioned features and those also listed here can each be used by themselves or several of them in any combination. Functionally similar or identical components are provided to some extent with the same reference symbols. The terms “left”, “right”, “top” and “bottom” used in the description of the exemplary embodiments relate to the drawings in an orientation with normally readable figure designation or normally readable reference symbols. The embodiments shown and described should not be understood to be conclusive but have an exemplary character for explaining the invention. The detailed description is used to inform the expert which is why known circuits, structures and methods are not shown in detail or explained in the description in order not to impede the understanding of the present description.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of one or more preferred embodiments when considered in conjunction with the accompanying drawings.